Method for Synchronizing Content-Dependent Data Segments of Files

ABSTRACT

In one embodiment of the present invention, the synchronisation of data segments is particularly necessary for the use of small screens. A method and a device are disclosed for synchronising content-dependent first data segments of a first data file and content-dependent second data segments of a second data file, the first and second data segments being respectively output in sequence according to their chronological order in such a way that based on a predeterminable assignment rule for the first and second data segments, each second data segment is output together with a corresponding first data segment.

PRIORITY STATEMENT

This application is the national phase under 35 U.S.C. § 371 of PCTInternational Application No. PCT/EP2006/060808 which has anInternational filing date of Mar. 16, 2006, which designated the UnitedStates of America and which claims priority on German Patent Applicationnumber DE 10 2005 016 866.3 filed Apr. 12, 2005, the entire contents ofwhich are hereby incorporated herein by reference.

FIELD

At least one embodiment of the invention relates to a method and/or to adevice.

BACKGROUND

For many decades, television signals have been transmitted to televisionsubscribers terrestrially and via cable. In Europe, for example, the PALstandard is used for this purpose, and in North America the NTSCstandard. Since all television sets in the respective countries aresuitable for receiving the television signal encoded e.g. according tothe PAL standard, only one television signal needs to be transmitted fora television program. As a result, a transmission of a plurality ofdifferently encoded television signals of a television program, forexample as a function of an image size or a performance capability ofthe television sets, is not required.

Due to the current introduction of new terminal devices, for examplemobile terminals with small screens, it will no longer be possible inthe future to provide a single television signal of a television programfor all terminal devices. Furthermore it may also be desired to providedifferent television subscribers with television signals of differentquality. In this case, for example, a low-quality television signal, inmonochrome for example, is provided for a television subscriber who paysa low monthly fee, whereas for a higher fee a high-resolution colortelevision signal is supplied.

New types of data compression algorithms, such as a video codingstandard ITU-T H.264 for example, as well as metadata languages, such ase.g. XML (XML=Extended Markup Language), were developed in order tosatisfy these requirements. With the aid of said metadata languages,contents of a document and/or a data stream, such as of a television orvideo signal for example, can be described in a structured manner bymeans of an XML schema language definition. Based on this definition,information can be obtained as required, with the aid of whichinformation a scaling of the television or video signal for differenttelevision subscribers can be realized.

From the prior art, it is known to use XML-based descriptions of datastreams in which individual components of the data stream are describedor marked according to their syntactical meaning by means of BSD units(BSD=Bitstream Syntax Description) with freely addressable gBSD units(gBSD=generic Bitstream Description) with regard to different adaptationpossibilities. A detailed description can be found in document [1]listed at the end of this application. These descriptions of the datastream are used in order to adapt the data stream to the requirement,for example in terms of data rate or quality of a television signal, ofan end subscriber, such as e.g. a television subscriber. For thispurpose the description of the data stream can first be suitablytransformed, with the data stream then being adapted based on thetransformed description.

A Bitstream Binding Language (BBL) method, by which a synchronizationcan be performed, is known from document [4]. Since BBL is based onXPath during the fragmentation of metadata and a description of asection of the metadata using XPath accounts for a sizable quantity ofdata, a large overhead is generated by this method. A proprietary methodis proposed for the fragmentation of the media data, as a result ofwhich it is generally not made possible to synchronize different datatypes.

Requirements for streamable digital items are presented in document [2].These requirements are based on the gBSD description, a distributed anddynamic adaptation being proposed for metadata.

Document [3] discloses two methods in which an adaptation of datastreams for streaming applications can be performed with the aid of thegBSD description and XSLT description.

SUMMARY

At least one embodiment of the invention specifies a method or, as thecase may be, a device which enables content-related first and seconddata segments to be synchronized in a simple and standard-compliantmanner.

With the method of at least one embodiment for synchronizingcontent-related first data segments of a first data file andcontent-related second data segments of a second data file, the firstand second data segments will be output in each case sequentiallyaccording to their chronological sequence in such a way that, based on apredefinable assignment rule for the first and second data segments,each second data segment will be output together with an associatedfirst data segment.

By way of the method according to at least one embodiment of theinvention, it is made possible in a simple manner to synchronizecontent-related first and second data segments. The synchronization iscontrolled by way of the predefinable assignment rule. In this way,content-related data segments can be output together according to theirchronological sequence. This is of advantage, since the assignment iscarried out without the use of timestamps. In this context the term“output” is understood to mean that first and second data segments are,for example, transmitted, stored together or passed on to a processingunit.

Preferably, the predefinable assignment rule is formed in such a waythat a second data segment is assigned precisely to a first data segmentand a further assignment is carried out following a number ofsequentially succeeding first data segments. In this way the assignmentrule can be described merely by specification of the number. This isadvantageous owing to the very low signaling overhead required forsignaling the number.

In an alternative embodiment, the predefinable assignment rule is formedin such a way that for each second data segment a first data segment isassigned by specification of a number, the number representing aposition of the first data segment within the chronological sequence ofthe first data segments. By this, the assignment of first and seconddata segments is achieved in a particularly flexible manner. In thisembodiment a plurality of second data segments can be assigned to afirst data segment.

Preferably, content-related first data segments and content-relatedsecond data segments of a plurality of second data files can besynchronized on the basis of a combined predefinable assignment rule. Bythis, the second data segments of a plurality of data files can beoutput in a synchronized manner. The combined predefinable assignmentrule enables a very compact and storage-efficient description.

With the method according to at least one embodiment of the invention,media data, in particular video or audio data, and/or metadata, inparticular encoded according to the XML standard, are described by wayof the first and/or second data file. In this way it is made possible byway of the method according to at least one embodiment of the inventionto synchronize a plurality of different data types.

The first and/or second data segments are preferably generated on thebasis of the generic bit stream syntax description or on the basis ofthe extensible stylesheet language transformation. With thisstandardized approach to generating the first and/or second datasegments it is possible to achieve a cost-effective, reliable andstandard-compliant implementation of the method according to at leastone embodiment of the invention, since the implementation can make useof standardized modules.

Furthermore, it is beneficial in practice to determine, on the basis ofa content-related marker, which first and second data segments will beoutput first in time. This enables the first and second data files to beaccessed at random. In particular a scene change in an image sequence ora change of speaker in a speech sequence is identified by way of thecontent-related marker. This enables a desired section of a speech orimage sequence to be located quickly, since the access is performed in acontent-based manner.

At least one embodiment of the invention also relates to a device forsynchronizing content-related first data segments of a first data fileand content-related second data segments of a second data file,comprising a synchronization module for outputting first and second datasegments in each case sequentially according to their chronologicalsequence, with each second data segment being output together with anassociated first data segment based on a predefinable assignment rulefor the first and second data segments. The method according to at leastone embodiment of the invention can be performed with the aid of thedevice.

BRIEF DESCRIPTION OF THE DRAWINGS

Further details and advantages of the invention are explained in greaterdepth with reference to FIGS. 1 to 5, in which:

FIG. 1 shows a device for performing the method according to anembodiment of the invention;

FIG. 2 shows the contents of a fragmentation description file using theXML description language, wherein a fragmentation of a first and/orsecond data file is described by way of the fragmentation descriptionfile;

FIG. 3 shows a tree diagram for the pictorial representation of anassignment rule;

FIG. 4 shows the contents of an assignment rule formed with the aid ofthe XML description language; and

FIG. 5 shows a chronological arrangement of synchronized first andsecond data segments S1, S2.

Elements having the same function and the same mode of operation areprovided with the same reference symbols in FIGS. 1 to 5.

DETAILED DESCRIPTION OF THE EXAMPLE EMBODIMENTS

The method according to the invention is explained in more detail belowwith reference to an example embodiment according to FIG. 1. In thisexample a video data stream and an audio data stream are to besynchronized. The video data stream has been compressed for exampleaccording to the video coding standard MPEG2, MPEG4 or ITU H.264. Saidvideo data stream is identified in FIG. 1 as the first data file D1. Theaudio data stream has been coded for example using the audio codingstandard ITU G.723.1 or MPEG4-AAC. The audio data stream is designatedin FIG. 1 by way of a name second data file D2.

For every first and second data file D1, D2 there exists in each case afragmentation description file D1_BSD, B2_BSD. Said fragmentationdescription file D1_BSD, B2_BSD specifies how content-related first andcontent-related second data segments S1, S2 can be obtained from therespective first and second data file D12, D2. The term“content-related” is understood to mean that first and second datasegments have a syntactical meaning within the respective data file,such as e.g. a plurality of succeeding second data segments representthe speech signal of a specific speaker within a dialog sequence.Furthermore, sections of the data file can also be understood thereby,such as e.g. individual images of a video sequence. In the followingdescription data segments always refer to content-based data segments.

FIG. 2 shows a first fragmentation description file D1_BSD for a firstdata file D1 encoded according to the MPEG4 standard. Data of a firstdata file D1 of this type is also known as media data MID. The firstfragmentation description file D1_BSD uses a gBSD syntax (gBSD=genericbit stream syntax description) for representing the fragmentation rules,the gBSD syntax being based on the XML language. Thus, for example, theline identified by a reference symbol X1 shows that an intracoded videoobject plane (I-VOP) begins at a byte address 26 within the first datafile D1 and comprises 2877 bytes. A detailed description of the mode ofoperation of the gBSD language is known from [1], the firstfragmentation description file D1_BSD being virtually identical withFIG. 9 from [1]. The extensible stylesheet language transformation (XSLTransformation or XSLT) represents another option for describing thefragmentation rules. This enables metadata MTD, in particularXML-encoded metadata, to be fragmented. Further information on XSLT canbe found in document [1].

According to FIG. 1, the first data file D1 and the first fragmentationdescription file D1_BSD are supplied to a fragmentation module MF. Thistakes first data segments S1 from the first data file D1 according tothe fragmentation rule of the first fragmentation description fileD1_BSD. Said first data segments S1 are passed by the fragmentationmodule MF to a synchronization module MS in the same chronologicalsequence R1 in which they are to be output. By the first data segmentsS1 is meant data segments of the first data file D1 which have acontent-related meaning for the first data file D1, such as e.g.individual images in a video sequence or individual spoken words in anaudio sequence.

In this case the chronological sequence identifies that sequence ofindividual first data segments S1 in which, for example, the latter areplayed back visually or made audible. For example, one data segmentcontains the word “Good” and a further data segment the word “Morning”,so the data segment containing “Good” is output first, and then the datasegment containing “Morning”. This chronological sequence R1 isrepresented symbolically in FIG. 1 by an arrow with the reference symbolR1. Additionally printed in the individual first data segments S1 arenumerals which indicate said chronological sequence R1. Thus, the firstdata segment with the numeral 0 is output first, then the one with thenumeral 1, and finally the first data segment with the highest numeral,i.e. with the numeral 9. The use of these numerals has been chosensolely to illustrate the method according to the invention more clearlyand does not constitute a requirement for the implementation of themethod according to an embodiment of the invention.

A similar procedure to that described above is followed in order toimplement the fragmentation of the second data file D2 by way of thesecond fragmentation description file D2_BSD and hence to output thecontent-related second data segments S2 in the chronological sequenceR2.

In a next processing step of the method according to an embodiment ofthe invention, the first and second data segments S1, S2 aresynchronized on the basis of a predefinable assignment rule Z. FIG. 3shows a possible layout of an assignment rule Z of this type in the formof a tree structure. The tree structure shows dependencies within theassignment rule Z indicated by means of dashed or unbroken connectinglines on the one hand and, on the other, keywords for using saidassignment rule Z in the form of an XML-based description. A start ofthe assignment rule Z can be identified in an XML-based document, suchas can be seen e.g. in FIG. 4, by an XML start word“StreamingInstructions”.

This is followed by the first data file D1 with an XML name“SynchronizationSource” and optionally the fragmentation rule byspecification of the first fragmentation description file D1_BSD with anXML name “FragmentationInstruction”. Alongside this, at least one seconddata file D2 with an XML name “SynchronizationTarget” is defined whichoptionally comprises the fragmentation rule in the form of the secondfragmentation description file D2_BSD with an XML name“FragmentationInstruction”. For each second data file D2,synchronization information SI is specified by means of an XML name“SynchronizationInstruction”, which defines the actual assignment of thefirst and second data segments S1, S2.

According to an embodiment of the inventive method, the synchronizationinformation SI can assume two different forms.

-   a) Thus, for example, the second data segment S2 is assigned a first    data segment S1 and a further assignment is carried out after a    number A of output, sequentially succeeding first data segments S1.    If the number A=2, this means that in each case the second first    data segment S1 is to be output with the second data segment S2.    This constellation is illustrated in FIG. 1. The first and second    data segments S1, S2 are output in their chronological sequence R1,    R2 by the synchronization module MS at the latter's data output G.    First, the respective first content-related first and second data    segments S1, S2 with the numeral 0 are forwarded. Since the number    is A=2, the next first data segment S1 with the numeral 1 is now    output. Next, a first and a second data segment S1, S2 with the    numeral 2 and 1 respectively are in each case output together at the    data output G. Finally, the first data segment S1 with the numeral 9    and the second data segment with the numeral 4 are output. This    procedure is identified in FIGS. 3 and 4 by means of an XML name    “SimpleSynchronize” and a reference symbol Z1. In FIG. 4 the value    of the number A is for example A=16.-   b) The use of the number A firstly enables only one single second    data segment S2 to be synchronized with a first data segment S1.    Secondly, the intervals at which a first and a second data segment    S1, S2 can be output together are permanently predefined by the    number A.    -   This means that in an alternative embodiment the predefinable        assignment rule Z can be formed in such a way that for each        second data segment a first data segment S1 is assigned by        specification of a number A, where the number N represents a        position of the first data segment S1 within the chronological        sequence R1 of the first data segments S1.    -   This alternative is explained in more detail with the aid of an        example according to FIG. 5. The first and second data segments        S1, S2 are to be output in the arrangement according to FIG. 5.        For this purpose the assignment rule Z can include the following        rule: V={1, 1, 2, 8, 8}. This means that the first and second        content-related second data segment S2, i.e. those with the        numerals 0 and 1, are to be synchronized with the second        content-related first data segment S1, which bears the        numeral 1. Furthermore, the next second data segment S2 with the        numeral 2 is to be output together with the first data segment        S1 with the numeral 2. Finally, the fourth and fifth second data        segment S2 with the numerals 3 and 4 are to be output together        with the first data segment with the numeral 8. Accordingly, the        above rule V={1, 1, 2, 8, 8} describes for each second data        segment S2, and moreover in the chronological sequence R2, the        chronological sequence R2 corresponding to the reading out of        said rule V from left to right, for which first data segment S1        a synchronization has to be performed. In this case the number        within the rule corresponds to the number N, which corresponds        in this exemplary embodiment to the numeral of the first data        segment S1.    -   The rule is presented somewhat differently in FIG. 4. In this        case the rule for the chronological sequence R2 applies not from        left to right, but from top to bottom. This means that the first        two second data segments S2 are output with the first data        segment S1 with the number 0, the following further four second        data segments S2 are assigned to the first data segment S1 with        the number 1 and are then output together in each case. This        variant of the assignment rule Z is marked in FIG. 3 by way of a        reference symbol z2 or an XML name “ComplexSynchronize”. The        respective number N in the XML document according to FIG. 4 can        be identified with the aid of an XML name “Occurance”, where        said number N can occur several times, as shown in the above        rule V.

The first and second data segments S, S2 output at the data output G caneach be sent with an RTP packet (RTP=Realtime Transport Protocol), theRTP packets of jointly output first and second data segments S, S2 beingprovided with an identical timestamp (“time code”). In an alternative,first and second data segments S1, S2 output together in each case canbe grouped in a respective RTP packet.

In an alternative extension of the method according to an embodiment ofthe invention it can be beneficial to determine the first and seconddata segments S1, S2 to be output first in time in each case within thefirst and second files D1, D2 on the basis of a marker M. The marker Mindicates which first or second data segment S1, S2 is to be read outfrom the associated first or second data file D1, D2. Thus, the marker Mcan be stored within the first or second fragmentation description fileDL_BSD, D2_BSD in such a way that the fragmentation module MF canunequivocally assign the right position within the associated data fileD1, D2. In the example according to FIG. 4, a first and a second markerM1, M2 are inserted, the first marker M1 being identified by means of atext “Start_1” and the second marker by way of a text “Start_2”. In thisway the first marker M1 for example is transmitted by thesynchronization module MS to the fragmentation module MF, whereupon thefragmentation module MF starts forming the first data segment S1 in theline with the reference symbol X1. If the second marker M2 istransmitted, the first data segment S1 begins to be created in the linewith the reference symbol X2.

In the previous examples, two data files D1, D2 to be synchronized wereconsidered. In general, more than one single second data file D2 can besynchronized with the first data file D1. For example, there are twosecond data files D21, D22. For this purpose a combined predefinableassignment rule ZK is formed. A combined assignment rule ZK of this kindis illustrated in the example according to FIG. 4. A first descriptionpart B_D1 contains information relating to the first data file D1 or itsfirst data segments S1. A second description part B_D21 includessynchronization information for a first second data file D21 or itssecond data segments S2. A third description part B_D22 includessynchronization information for the further second data file D22 or itssecond data segments S2. The second description part B_D21 was formedaccording to the first assignment rule Z1, and the third descriptionpart B_D22 according to the second assignment rule Z2.

Example embodiments being thus described, it will be obvious that thesame may be varied in many ways. Such variations are not to be regardedas a departure from the spirit and scope of the present invention, andall such modifications as would be obvious to one skilled in the art areintended to be included within the scope of the following claims.

DOCUMENTS

-   [1] Panis G. et al., “Bitstream Syntax Description: A Tool for    Multimedia Resource Adaptation within MPEG-21”, Signal Processing,    Image Communication, Elsevier Science Publishers, Amsterdam, NL,    Volume 18, No. 8, September 2003, pages 721-747, ISSN: 0923-5965-   [2] S. Devillers et al., “Metadata streaming use case and    requirements”, ISO/IEC JTC 1/SC 29/WG11 M11638, Hong Kong, January    2005-04-08-   [3] H. Hellwagner et al., “Report of CE on Bit-Stream Adaptation in    Constrained and Streaming Environments”, ISO/IEC JTC 1/SC 29/WG11    M11706, Hong Kong, January 2005-   [4] I. Burnett et al., “Bitstream Binding Language—A Grammar for    Digital Item Transport”, ISO/IEC JTC 1/SC 29/WG M11611, Hong Kong,    January 2005

1. A method for synchronizing content-related first data segments of afirst data file and content-related second data segments of a seconddata file, the method comprising: sequentially outputting the first andsecond data segments according to their chronological sequence in such away that each second data segment is output together with an associatedfirst data segment on the basis of an assignment rule for the first andsecond data segments.
 2. The method as claimed in claim 1, wherein theassignment rule is formed in such a way that a second data segment isassigned to precisely one first data segment and a further assignment iscarried out after a number of sequentially succeeding first datasegments.
 3. The method as claimed in claim 1, wherein the assignmentrule is formed in such a way that, for each second data segment, a firstdata segment is assigned by specification of a number, the numberrepresenting a position of the first data segment within a chronologicalsequence of the first data segments.
 4. The method as claimed in claim1, wherein content-related first data segments and content-relatedsecond data segments of a plurality of second data files aresynchronized on the basis of a combined assignment rule.
 5. The methodas claimed in claim 1, wherein media data are represented by at leastone of the first and second data file.
 6. The method as claimed in claim1, wherein at least one of the first and second data segments aregenerated on the basis of at least one of the generic bit stream syntaxdescription and the extensible stylesheet language transformation. 7.The method as claimed in claim 1, further comprising: determining, basedupon a content-related marker, which first and second data segments areto be output first in time.
 8. The method as claimed in claim 7, whereinat least one of a scene change in an image sequence and a change ofspeaker in a speech sequence is identified by way of the content-relatedmarker.
 9. A device for synchronizing content-related first datasegments of a first data file and content-related second data segmentsof a second data file, comprising: a synchronization module to outputfirst and second data segments, sequentially, according to theirchronological sequence, each second data segment being output togetherwith an associated first data segment based upon of an assignment rulefor the first and second data segments.
 10. The method as claimed inclaim 2, wherein content-related first data segments and content-relatedsecond data segments of a plurality of second data files aresynchronized on the basis of a combined assignment rule.
 11. The methodas claimed in claim 3, wherein content-related first data segments andcontent-related second data segments of a plurality of second data filesare synchronized on the basis of a combined assignment rule.
 12. Themethod as claimed in claim 5, wherein the media data include at leastone of video and audio data.
 13. The method as claimed in claim 1,wherein at least one of media data and metadata are represented by atleast one of the first and second data file.
 14. The method as claimedin claim 13, wherein the metadata include metadata encoded according tothe XML standard.
 15. The method as claimed in claim 13, wherein themedia data include at least one of video and audio data.
 16. The methodas claimed in claim 1, wherein the assignment rule is predefinable. 17.The device as claimed in claim 9, wherein the assignment rule ispredefinable.